1. Field of the Invention
This invention relates to omnidirectional image sensing with reference to a single viewpoint, and, more particularly to such image sensing using a truncated, substantially paraboloid-shaped reflector.
2. Discussion of the State of the Art
For many applications such as surveillance, teleconferencing, remote sensing, photogrammetry, model acquisition, virtual reality, computer graphics, machine vision and robotics, it is desirable that an imaging system have a large field of view so as to be able to take in as much information as possible about the world around it.
Traditional imaging systems include a camera with a lens that provides a perspective projection of an image. However, a camera with even a very wide angle lens has only a limited field of view (i.e., covering less than a full hemisphere). This limited field of view may be expanded by tilting and panning the entire imaging system about its center of projection. One such system is described in S. E. Chen, "Quicktime VR--An Image-Based Approach to Virtual Environment Navigation", Proc. of SIGGRAPH 95, (8):29-38, August (1995). The article by L. McMillan and G. Bishop, "Plenoptic Modeling: An Image-Based Rendering System", Computer Graphics: Proc. of SIGGRAPH, August 1995, pp. 39-46, also describes a traditional pan-and-tilt system. This type of system has two serious drawbacks, however, one being the obvious disadvantages associated with a device having critical moving parts, and the second being the significant amount of time required to make a full rotation in order to view the surrounding world. This time limitation makes such a device unsuitable for real-time applications.
Another approach to increasing the field of view in an imaging system is by employing a so called "fish eye" lens as is disclosed in E. L. Hall et al., "Omnidirectional Viewing Using a Fish Eye Lens", SPIE Vol. 728 Optics, Illumination, and Image Sensing for Machine Vision (1986), p. 250. Since the fish eye lens has a very short focal length, the field of view may be as large as a hemisphere. The use of such lenses in an imaging system is problematic, however, in that they are significantly larger and more complex than conventional lenses. Moreover, it has been difficult to develop a fish eye lens with a fixed viewpoint for all points of the relevant scene. U.S. Pat. No. 5,185,667 to Zimmerman, and U.S. Pat. No. 5,359,363 to Kuban et al. are also directed to the use of fish eye lenses to replace conventional pan and tilt mechanisms, and accordingly suffer from the same disadvantages.
Other prior art devices have used reflecting surfaces to increase the field of view. One such prior art device is disclosed in V. S. Nalwa, "A True Omni-Directional Viewer", AT&T Bell Laboratories Technical Memorandum, BL0115500-960115-01, January 1996. Nalwa discloses the use of multiple planar reflecting surfaces in conjunction with multiple charge coupled device ("CCD") cameras to obtain a 360 degree panoramic image of a 50 degree band of a hemispherical scene. Specifically, in Nalwa, four planar mirrors are arranged in the shape of a pyramid, with one camera being positioned above each of the four planar reflecting sides, and with each camera viewing slightly more than 90 degrees by 50 degrees of the hemispherical scene. This system suffers from the serious drawback of requiring multiple sensors to capture a 360-degree image. In addition, this system suffers from the inherent problems associated with distortion at the "seams" when the separate images are combined to provide a full 360 degree view.
Curved reflective surfaces have also been used in conjunction with image sensors. Both Yagi et al., "Evaluating Effectivity of Map Generation by Tracking Vertical Edges in Omnidirectional Image Sequence", IEEE International Conference on Robotics and Automation, June 1995, p. 2334, and Yagi et al., "Map-Based Navigation for a Mobile Robot With Omnidirectional Image Sensor COPIS", IEEE Transactions on Robotics and Automation, Vol. II, No. 5, Oct. 1995, disclose a conical projection image sensor (COPIS) which uses a conical reflecting surface to gather images from the surrounding environment, and processes the information to guide the navigation of a mobile robot. Although the COPIS is able to attain viewing in 360 degrees, it is not a true omnidirectional image sensor because the field of view is limited by the vertex angle of the conical mirror and by the viewing angle of the camera lens. Furthermore, the COPIS does not have a single viewpoint, but instead has a locus of viewpoints lying on a circle. This locus of multiple viewpoints causes distortion in the captured images, which cannot be eliminated to obtain pure perspective images.
Yamazawa et al., "Obstacle Detection With Omnidirectional image Sensor Hyperomni Vision", IEEE International Conference on Robotics and Automation, October 1995, p. 1062, discloses a purported improvement in the COPIS system which involves the use of a hyperboloidal reflecting surface in place of a conical surface. As discussed therein, the rays of light which are reflected off the hyperboloidal surface, no matter where the point of origin, will all converge at a single point, thus enabling perspective viewing.
Although the use of a hyperboloidal mirror is advantageous in that it enables full perspective image sensing, because the rays of light which make up the reflected image converge at the focal point of the reflector, positioning of the sensor relative to the reflecting surface is critical, and any disturbance will impair the image quality. Further, the use of a perspective-projection model inherently requires that as the distance between the sensor and the mirror increases, the cross-section of the mirror must increase. Therefore, practical considerations dictate that in order to keep the mirror at a reasonable size, the mirror must be placed close to the sensor. This in turn causes complications to arise with respect to the design of the image sensor optics. In addition, mapping sensed image to usable coordinates requires complex calibration due to the nature of the converging image. A further drawback is that the relative positions of the mirror and the optics cannot be changed while maintaining a single viewpoint. Thus, a hyperboloidal mirror system cannot take advantage of the relative movement of the mirror and optics to adjust the field of view of the system, while maintaining a single viewpoint.
Prior to Yamazawa et al., U.S. Pat. No. 3,505,465 to Donald Rees also disclosed the use of a hyperboidal reflecting surface. Accordingly, the Rees disclosure also suffers from the same drawbacks as that of Yamazawa et al.
The above-described prior art devices fail in one of two ways. They either fail to provide a truly omnidirectional imaging apparatus that is capable of sensing a scene from a single viewpoint, making it impossible to provide distortion-free images with the apparatus, or they provide apparatus that require complex calibration and implementation.